A Unified View of Spectral Clustering∗

We formulate a discrete optimization problem that leads to a simple and informative derivation of a widely used class of spectral clustering algorithms. Regarding the algorithms as attempting to bi-partition a weighted graph with N vertices, our derivation indicates that they are inherently tuned to tolerate all partitions into two non-empty sets, independently of the cardinality of the two sets. This approach also helps to explain the difference in behavior observed between methods based on the unnormalized and normalized graph Laplacian. We also give a direct explanation of why Laplacian eigenvectors beyond the Fielder vector may contain fine-detail information of relevance to clustering. Another advantage of our discrete formulation is that it admits a random graph interpretation, showing that spectral clustering may be viewed as maximum likelihood partitioning under the assumption that the data is an instance of a graph with random edge weights. The resulting distribution on the weights formalizes and quantifies the intuitive notion that vertices in the same cluster are more likely to have high weights than vertices in different clusters. Numerical experiments that illustrate the analysis are included. keywords: balancing threshold, Rayleigh-Ritz Theorem, Fiedler vector, graph Laplacian, random graph, maximum likelihood, partitioning. AMS Subject Classification: 65F15, 90C27, 05C85 This manuscript appears as University of Strathclyde Mathematics Research Report 02 (2004). Department of Mathematics, University of Strathclyde, Glasgow G1 1XH, UK. Supported by Research Fellowships from The Leverhulme Trust and The Royal Society of Edinburgh/Scottish Executive Education and Lifelong Learning Department. Department of Mathematics, University of Turku, FIN-20014 Turku, Finland. Supported by the Academy of Finland, under grant number 53441.